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ABSTRACT 



The Physical Therapy Department of Nova Southeastern 
University (NSU PT) changed its curriculum to problem-based learning, and in 
conjunction with changes needed new evaluation forms for accreditation and 
program requirements. The purpose of this practicum study was the development 
and validation of faculty and course evaluation forms for students at NSU PT. 
The process began with a literature review and a review of the NSU PT 
mission, philosophy, and accreditation documents. An internal formative 
committee survey was conducted, and evaluation forms were developed and 
validated by the summative evaluation committee of four faculty members. 

Items with a mean Likert score of 3.5 and above were accepted, and items 
below this score were reconsidered, redesigned, or excluded. Editorial 
recommendations were incorporated to make a final product of 5 Likert- scale 
forms, with 7 to 14 items each, considering affective, behavioral, 
psychomotor, and cognitive areas with an open-ended comment section. These 
forms were developed: (1) student self-assessment for the course; (2) student 

self-assessment for the laboratory; (3) student performance assessment for 
the clinic; (4) faculty tutor assessment for the course; and (5) faculty 
tutor assessment for the laboratory work. Recommendations are to implement 
these forms, conduct a pilot test, and consider longitudinal studies to track 
curriculum progress and aid in faculty and student development. Five 
appendixes include NSU PT mission statements, a list of committee members, 
the formative committee questionnaire, the summative committee questionnaire, 
and the five forms of the evaluation product. (Contains 3 tables and 24 
references . ) ( SLD) 
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Abstract of a developmental practicum report presented to Nova Southeastern 
University in partial fulfillment of the requirements 
for the degree of Doctor of Education 

DEVELOPMENT AND VALffiATION OF FACULTY AND COURSE EVALUATIONS 
FOR STUDENTS FOR NOVA SOUTHEASTERN UNIVERSITY 
PHYSICAL THERAPY PROGRAM 
by 

Gina M. Musolino 
July, 1997 

Nova Southeastern University Physical Therapy (NSU PT) transitioned the curriculum 
to Problem-Based Learning (PBL) and required revision of evaluation forms for 
accreditation and program requirements. The purpose of this study was the development 
and validation of faculty and course evaluations for students for NSU PT. The research 
question addressed was: What are the appropriate and valid contents and design of faculty 
and course evaluation forms that are representative of PBL for NSU PT? 

Procedures used in this developmental methodology to collect data and address these 
questions included (a) a literature review and review of NSU PT mission, philosophy and 
accreditation documents, (b) internal formative committee survey, (c) establishment of 
design and content of the evaluation forms project based on synthesis of procedures a and 
b, (d) external summative committee validation, (e) data compilation summary, (f) revision 
of product, and (g) presentation of the product with recommendations for implementation. 

Utilizing the procedures, evaluation forms for NSU PBL PT were developed and 
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validated. Items with a mean Likert score of 3.5 or above, were accepted, items below this 
score were reconsidered, redesigned or excluded. Editorial recommendations were 
incorporated and the final product consisted of 5 Likert-scale forms, with 7-14 items 
considering affective, behavioral, psychomotor, and cognitive areas with an open ended 
comment section. Recommendations were to implement the evaluation forms product, 
conduct a pilot test, consider longitudinal studies to track curriculum progress and aid in 



faculty and student development. 
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Chapter 1 
INTRODUCTION 

Nova Southeastern University (NSU) is a private university offering traditional and 
non-traditional programs of study from kindergarten through the doctoral level. The 
Physical Therapy Department is a component of the Health Professions Division at NSU as 
a result of a recent merger of Nova and Southeastern Universities. The physical therapy 
program was a new program seeking accreditation from the American Physical Therapy 
Association (APTA). The program, in the first year, consisted of a traditional curriculum, 
but in the second year, altered the androgogy to one of Problem-Based Learning (PBL). 

Nature of the Problem 

Nova Southeastern University’s Physical Therapy Program has shifted the overarching 
androgogical approach of education from traditional methods of instruction to Problem- 
based Learning (PBL) beginning with the 1995 Fall term. Evaluation of educational 
programs and curricula is an essential process in professional education, and an 
accreditation requirement. The problem was current faculty and course evaluation forms 
required revision, due to the instructional shift to PBL. The revision in forms was required 
to reflect this PBL process of learning and teaching and for accreditation. 

Purpose of the Study 

The purpose of this study was the development and validation of faculty and course 
evaluations for students for Nova Southeastern University Physical Therapy Program (NSU 
PT). The revision was needed for faculty evaluation forms for student assessment of 
courses that would reflect Problem-Based Learning (PBL). PBL androgogy is a unique 
instructional format, requiring a different evaluation form than traditional methods. No 
lectures occur in PBL, therefore most traditional evaluation forms did not meet the needs 
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for PBL evaluation. Curricula of this nature create new demands for the evaluation of the 
faculty and courses (Hay, 1996). 

Significance to the Institution 

The Physical Therapy (PT) Program at Nova Southeastern University (NSU) was a 
developing program seeking accreditation from the American Physical Therapy Association 
(APTA). The program reached CANDHIACY (APTA, 1996) status with the Commission 
on Accreditation for Physical Therapy Education (CAPTE). A portion of the Evaluative 
Criteria, for Accreditation of Education Programs, for the Preparation of Physical 
Therapists, requires; in Section One, Organization, of the Self Study Report Format the 
faculty evaluation forms and examples of their use on-site. The former faculty evaluation 
forms did not reflect Problem-Based Learning (PBL) and were representative of traditional 
methods of instruction (i.e. lectures). 

The feasibility of this study was significant as faculty and course evaluation tools are an 
accreditation requirement for the CAPTE, Southern Association of Colleges and Schools 
(SACS), and NSU policy. SACS (1996) delineates in Section HI, Institutional Effectiveness, 
criteria 3.1 Planning and Evaluation: Educational Programs, that a broad based systematic 
approach to planning and evaluation of teaching, research and service must occur and be 
appropriate to the institution. SACS guidelines state that the results of these evaluations 
must be used to improve educational programs, services and operations; and that the 
institution must develop procedures and guidelines to evaluate educational effectiveness 
with both quantitative and qualitative data gathering. Evaluation of instructional delivery 
is one example of this SACS criteria (pp 20-21). This project provided the beginnings of 
appropriate faculty and course evaluation forms for the PBL curricula for NSU PT. 
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According to West & Watson (1996), ^Troblem-based learning strategies can be intricately 
interwoven into a complex core curriculum to meet accreditation criterion, while at the 
same time increasing self-directed learning with students” (p 2). 

Relationship to Seminar 

This practicum was directly related to the Governance and Management seminar in the 
purview of the principles of structural frameworks. Structural frames are concepts derived 
from both management and sociology, ‘‘emphasizing goals, specialized roles, and formal 
relationships. Structures — commonly depicted by organization charts — are designed to fit 
an organization’s environment and technology” (Bolman & Deal, 1997, p 13). The 
Governance structural framework, imposed by accreditation, guides the need for faculty 
and course evaluation forms, to assist in meeting the organizational goals. The standards 
and policies determined by CAPTE (1996) and SACS (1996) provide the direction for the 
need for evaluation assessments. 

Considering the human resource framework, according to Bolman and Deal (1997), the 
process of evaluation is a “basis for helping individuals grow and improve” (p 267). The 
challenge of accreditation is to keep on “top of large, complex sets of activities; set goals 
and policies under conditions of uncertainty and attain intellectual grasp of policy issues” 

(p 277). The evaluation project entailed both the structural and human resource frames. 

Relationship to Concentration 

The area of concentration was Curriculum Development and Evaluation. The 
evaluations developed were part of the curriculum review process, an integral portion of 
program development. One piece of curriculum improvement was the feedback loop of 
information from formative and summative evaluations. Selecting appropriate items that 
reflected the department and institutional criteria for effectiveness, was a necessary 
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component for evaluation and curriculum review (Davis, 1993). In problem-based 
learning, curricular goals were assessed through several methods, self-assessment, team 
assessment and faculty/tutor assessment, to provide constructive feedback for improvement 
(Barrows, 1997). 

Research Questions 

The research questions for this study were, ‘^What are the appropriate and valid 
contents of faculty and course evaluation forms, that are representative of problem-based 
learning (PBL), for Nova Southeastern University Physical Therapy Program (NSU PT)?” 
and “What are the appropriate and valid design of faculty and course evaluation forms, 
that are representative of PBL, for NSU PT?” 

Definition of Terms 

For purposes of this practicum, the following terms were defined. 

A ppropriate . Items which are suitable for meeting the purpose of course evaluations. 

Faculty and course evaluation form . A scannable paper pencil instrument which 
provides the student consumer the opportunity to assess faculty and course effectiveness in 
the NSU PT PBL curriculum. 

Problem-Based Learning (PBLI . PBL is a curriculum innovation characterized by the 
use of case studies as a vehicle through which small groups of students learn problem- 
solving skills while simultaneously directing their own acquisition of content knowledge 
(Albanese & Mitchell, 1993). Upon presentation of a case, students decide how to identify 
major problems, gaps in their knowledge and skills, and strategies for resolving these gaps. 
Learning is motivated by a need to resolve problems (Barrows, 1997). 

Team teaching . A method of instruction where faculty participate as part of a group in 
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facilitating PBL; faculty serve as faculty team leaders, advisors, mentors, tutors and 
academic coordinators for clinical education for groups of professional students for NSU 
PT. 

Tutor . “Faculty member who facilitates a learning process, guiding rather than 
directing learning, and being expected to consistently provide constructive verbal feedback 
regarding both content knowledge and the learning process” (Hay, 1996, p 23). 

Valid . The items are appropriate for the course evaluations for NSU PBL PT. Both 
content and face validity applications were considered. Content validity: the criteria are 
accurate and timely in both substance and presentation. Face validity: criteria appear to 
be useful and attractive to the target audience. 
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Chapter 2 



REVIEW OF RELATED LITERATURE 
Overview 

The literature review provided an overview of the concept and purposes of evaluation, 
design types and content criteria for evaluation in problem-based learning (PBL). 

Extensive literary searches were completed for this project, in the ERIC, CINHAL, and 
Medline databases, electronic search engines and textbooks. 

Central to the preparation of professionals is a graduate who has the ability to function 
in a rapidly changing health care environment. Professionals need to solve problems and 
make decisions in ambiguous situations. Traditional conceptions of evaluation are 
frequently based on a process that links assessment tools directly to program goals and 
objectives. Evaluation forms of this type, focus on achievement of the goals or objectives, 
but do not begin to examine the process of learning, the experience of students, assessment 
of whether the goals are worth achieving, or the influence of the implicit curriculum. The 
evaluation should not be conceived only as whether the goals have been achieved, but the 
social and cultural context of the educational experience. 

Evaluation 

Purposes of Evaluation 

Student and faculty evaluation, according to Astin (1991), is carried out for the purpose 
of “supporting the resources and reputational views of excellence and to support certain 
administrative practices” (p 13). Courts and Mclnemey (1993) expounded upon the broad 
picture of assessment, “most consider a useful evaluation not an end unto itself but a means 
of diagnosing and correcting problems in the teaching/learning process” (p xv). Several 



sources concurred that evaluation and assessment should be directly tied to improving 
instruction and to improving the quality of the lives of students and teachers (Banta, Lund, 
Black, & Oblander, 1996; Eley & Stecher, 1997). Astin (1991) delineated that assessment 
of ‘‘faculty should enhance their performance as teachers and mentors of students and as 
contributors to the advancement of knowledge; advancing the educational mission, and the 
teaching and research functions of the university” (p 4). 

Astin further argued the point that the students’ perspective is rarely incorporated in 
evaluation and universities should at least be partially responsive to this perspective. Astin 
stated that this “almost forces us to tailor make part of our assessment to the particular 
needs and aspirations of the individual student” (p 40). This readily agrees with Stephen 
Feldman’s (Boyd, 1994) thought of students as customers of education; and Rensis Likert 
using “survey data to show that ‘employee centered’ supervisors, who focused more on 
people and relationships, typically managed higher-producing units than ‘job centered’ 
supervisors, who made decisions themselves and dictated to subordinates” (Bowman & 
Deal, 1997, p 139). 

Howard Barrows (1994), related the need for evaluation tools for the faculty, to 
evaluate the adequacy of their instructional program. Faculty need to know if there are 
common problems among the students that may reflect inadequacies in the curriculum or 
the teaching-learning approach that is being used. At McMaster University, students 
received feedback openly from tutors in small group sessions on an on-going basis, with 
recurrent formative evaluations. Students participated in self and faculty evaluations on 
an on-going basis (Bridges & Hallinger, 1995). 

The McMaster evaluation system was based on five principles: excerpt from. 
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Evaluation of Health Sciences Education Programs (Schmidt, Lipkin, deVries, & Creep, 
1993): 

1. Student performance should be measured against learning objectives. They must 
evaluate the student’s ability to identify and deflne health problems at an individual 
and community level.... The student must have the ability to develop the personal 
characteristics and attitudes required for a career in the health professions. 2. 
Evaluation methods should be compatible with the learning objectives. Evaluation 
should be carried out within the framework of problem-solving in which selection, 
understanding, and integration of concepts from a variety of disciplines are assessed. 3. 
Evaluation information should come from a variety of relevant sources....self- 
assessment, peer assessment, and assessment by the tutor and other faculty. 4. 
Evaluation should be ongoing. Immediate feedback and ongoing evaluation permits 
the student to identify and correct areas of weakness, thereby strengthening and 
reinforcing the learning process. 5. Evaluation is a shared responsibility. Students in a 
self-directed learning program share in the accountability for evaluation. It is not just 
an assessment imposed and controlled by faculty, (pp 166-169) 

Evaluation design criteria 

Evaluation of teaching should employ a variety of information sources, one source, in 
higher education, includes student responses to evaluation questionnaires. Well 
constructed questions are based on characteristics representative of teaching quality. 
Methods of phrasing the evaluation questions also determines the reliability and validity of 
the instrument. Historically, several question forms have been employed to glean faculty 
and course evaluation feedback. One format, Likert questionnaires, present a continuum 
for agreement, indicating the respondents’ values and attitudes. It was argued that this 
method does not gather information about the intended purpose, information on the 
teaching about which the students are responding (Eley & Stecher, 1997). However, in 
PBL this is one of the intentions of evaluation. 

Davis (1993) made the following design recommendations for evaluation 
questionnaires: using forms that give the students the opportunity to provide quantitative 
ratings, provision of a numeric rating scale for at least some items, including items that ask 
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students about the effects of the course, and quantitative measures of overall effectiveness 
with at least one open-ended item, and Anally, keeping the form short. Davis asserted that 
the use of numeric ratings aid in calculation of comparison over time and brevity of forms 
is needed due to the numbers of evaluations students may be completing (pp 397-404). 

Another alternative evaluation question form is the behaviorally anchored scale, 
comprising a ‘‘description of some performance element to be rated, together with a rating 
scale in which each scale point is deAned (anchored) by a description of some performance 
instance typical of someone rated at that level” (Eley & Stecher, 1997, p 66). A drawback 
is the difficulty in generation of scale items and expectation performances for each scale 
level. Considerable research comparing the two evaluation form design types, Likert, 
behaviourial anchored scales and others, have received mixed reviews. However, when 
utilized in the best scenario or intent, the assessment process increases student rapport and 
facilitates a climate for an educational partnership. Three words, according to Banta et al. 
(1996, p 291) described why these researchers have been successful with evaluations, trust, 
authenticity and responsiveness. The evaluations are anonymous and items are conducive 
to improving the learning process. The dialogue is opened up and responses to feedback 
are made with discussion, change and/or further explorations with faculty and students. 

Evaluation content criteria 

Astin (1991) suggested criteria should include student satisfaction in terms of academic 
"advising, quality of instruction, student-faculty contact (p 143) and process assessment, and 
Caching practices and styles (p 248). Hay (1996) agreed and contended that evaluation 
~1form criteria should consider the tutor with the AAowmg inclusion factors: “1. The ability 
to^re for students, 2. A knowledge of (^urse stnn^re and teaching philosophy, 3. The 
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ability to encourage independent thinking and 4. A knowledge of specific medical problems 
being studied” (p 22). 

Davis (1993) suggested, for content evaluation, utilizing items that reflect your 
department’s and institution’s criteria of effectiveness, and are within the student’s range 
of judgement and stating each item clearly (pp 397-404). Astin (1991) adjudicated the use 
of outcome data for evaluation content in both behavioral and affective domains. He 
suggested evaluating items pertaining to “psychological: values, interests, self-concept, 
attitudes, beliefs, satisfaction with the university;... and behavioral: leadership and 
interpersonal relations” (p 45). Astin (1991) further identified other potentially important 
affective outcomes including, “self-understanding, honesty, maturity, motivation for 
further learning, understanding of other peoples and societies, self-esteem, social 
responsibility and even good mental and physical health” (p 56). As health care providers, 
physical therapists must function in all three domains, cognitive, affective and psychomotor 
areas, while maintaining their own health. Therefore, it would seem prudent to evaluate all 
domains. The main limitation of evaluating the results of affective domain information is 
the students’ time required to produce information on a relatively small number of 
affective outcomes. Astin (1991) best described this as, “instruments have relatively high 
‘fidelity’ with a relatively narrow ‘band width’” ( p 58). However, this does not appear to 
be an adequate “excuse” to undemtilize affective evaluation measures. 

In contrast, Banta et al. (1996), noted specific behaviors that are amenable to change 
should be captured, students should not be asked to comment on things such as the tutors 
contemporary knowledge or how prepared students are for advanced courses, as students 
are not yet privy to make these judgements. Furthermore, the researchers consider 
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evaluation for the tutor/facilitator, a step further, delineating specific facilitator criteria 

into four distinctive categories, ‘‘genuine concern, encouragement, knowledgeable, 

approachable and easy to talk to” (p 289). The researchers suggested on the self evaluation 

level to assess whether or not the learner: “did the readings to prepare, wrote 

questions/comments pre tutor session, actively participated in tutor exercises and 

discussions, thought about how readings/case applied to their life and future job” (p 288). 

These researchers advised for the evaluation of the course that “stimuli (exercise, film, case) 

were linked well to the objectives, whether the stimuli were well prepared and organized 

and helped the learner to understand the topic better, and whether there was a good 

balance between theory and application of the topic” (p 289). Finally, the researchers 

proposed open comments and suggestions for the evaluation criteria. 

These suggestions for appropriate and valid contents and design for faculty and course 

evaluations were considered for this project. As student perspectives change, so to must 

the methods and course woii( offered by universities that wish to remain consumer- 

oriented in the business of education. This view is supported by Stephen Feldman (Boyd, 

1994), former president of NSU, describing the business of academia: 

Growth and expansion will only occur if NSU defines itself as a business, its students as 
customer and the world of education as the competition. If you manage things well, 
you will succeed. It doesn’t matter whether you’re producing widgets or professionals. 
(P 7) 

Research Considerations 
Introduction to Research Review 

The current literature considers both qualitative and quantitative critical inquiry for 
evaluation in PBL in health care and law. Pertinent to this project, research reviews were 
considered in the areas of outcome evaluation, tutor behavior, performance criteria, and 
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tasks, interviews and focus groups with student consumers and the use of portfolios. 

Research Review 

Schmidt & Moust (1995), compared student achievement with a causal model of the 
tutor behaviors influence in the context of PBL. The subjects were 524 undergraduate 
tutorial groups at the University of Limburg, from 1992-1993. These groups were guided 
by 261 tutors and Likert scale data was taken following sessions; with correlations drawn in 
terms of social congruence, subject matter expertise, cognitive congruence, tutorial group 
functioning, self-study time, intrinsic interest in the subject matter and student 
achievement. 

According to Schmidt & Moust (1995) the results of this study suggested that ‘‘subject- 
matter expertise, a commitment to students’ learning and their lives in a personal, 
authentic way; and the ability to express oneself in a language used by the students are all 
determinants of learning in a PBL curricula” (p 708). Shortcomings of the study were 
identified as problems with the students rating the tutor, as students were there to learn 
and students possessing an implicit theory of effectiveness, for a tutor, which may not be 
congruent with tutor expectations (pp 708-714). 

In another study, in health care education. Hay (1996) investigated whether a 16-item 
tutor performance evaluation was able to discern a consistent factor structure. The 
subjects were both occupational therapy (OT) and PT students enrolled in McMaster 
University, with PBL groups evaluating tutors. During a two year period, 364 evaluations 
were completed by 60 tutorial groups. Tutor groups were reassigned each term and 
tutorial size ranged from 7 to 10 students per group. Evaluations were completed pre 
grade assignment and the response rate was 100% for OT students and 94% for PT 
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students. Forms were scored using Likert scale data and factor analyses was completed. 

This statistical analysis, according to Hay, provided an explanation of the correlation 

between two or more variables in terms of some underlying construct or factor. The scale 

was found to have established reliability. Hay utilized the following items for evaluation: 

This tutor: 1. Provided clear insight into the objectives and expectations of the course, 
2. Discussed students’ expectations of the course, 3. Assisted students in planning to 
meet course objectives, 4. Encouraged effort and rewarded student contributions, 5. 
Consistently provided reasonable, constructive, verbal feedback, 6. Was objective in 
formal student evaluations, 7. Was sensitive to students’ learning needs, 8. Displayed 
concern and empathy for students, 9. Was enthusiastic about educational role, 10. 
Provided a reasonable degree of student autonomy, 11. Provided a reasonable degree of 
student autonomy in selecting learning strategies, 12. Communicated clearly with 
students, 13. Assisted in the development of individuals’ communication skills within 
the group, 14. Facilitated development of student problem solving skills, 15. Effectively 
assisted students to develop their reasoning skills, 16. Challenged students to develop 
their knowledge to an appropriate level, 17. Stimulated interest in course content, 18. 
Assisted the group in determining when an appropriate level of understanding had 
been reached, and 19. Overall rating of performance, (p 23) 

Interestingly, Hay did not tackle the areas of self-assessment nor tutor knowledge, due 
to the controversy on the ability of a student to adequately evaluate this aspect of the tutor. 
(Hay, 1996 and Dolmans, Wolfhagen, Schmidt & van der Vleuten, 1994). However, it 
would seem that if tutors are increasing knowledge, over time, the students would be 
capable of this aspect of tutor evaluation. 

Considering the graduates will work with physical therapists, a study by May, Moran, 
Lemke, Karst and Stone (1995) surveyed physical therapy clinical educators from 76 
clinical sites using the Delphi technique. The educators were asked to identify generic 
abilities critically important to physical therapy practice. These criteria were then 
developed into ability-based assessments for the University of Wisconsin-Madison PT 
program. The impetus for this study came from academic faculty inquiring with the 
clinical educators about why some students fail to make the transition from didactic 
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courses to the clinical internship. Generic abUities included the following in rank order: 

“1. Commitment to learning. 2. Interpersonal skills. 3. Communication skills. 4. 
Effective use of time and resources. 5. Use of constructive feedback. 6. Problem solving. 

7. Professionalism. 8. Responsibility. 9. Critical thinking. 10. Stress management”(p 4). 
These criteria are in agreement with the mission and philosophy of NSU PT (Appendix A) 
and the CAPTE standards. The disadvantage, noted by May et al. (1995) was that ability- 
based assessment of this nature focused more on outcome than content. However this is the 
direction being demanded by the consumer and employer. 

Dolmans et al. (1994) developed an instrument to assess tutor performance in PBL 
tutorial groups. Subjects consisted of 293 tutors and 18 tutorial group sessions over a 19 
week academic year, with students in first through fourth years. The average response rate 
was 81% and a pilot study, had been performed the prior academic year. Through the 
pilot study, it appeared the researchers employed a modified Delphi technique, but the 
researchers never clearly stated. During the pilot, 150 tutors and 100 students were 
provided a list of 16 tutor behavioral characteristics and asked to rate them as important 
indicators and whether or not the item was clearly stated. The pilot study resulted in 13 
statements: “6 items related to the tutor’s task to guide students through the learning 
process, 4 items about the tutor’s content knowledge input and 3 items about the tutor’s 
commitment to the group’s learning” (p 551). 

The items were presented with the options for the student to rate the tutor as 
insufficient, neutral, sufficient or not applicable. These numerical ranges were then 
assigned one of seven qualifications, such as extremely poor, sufficient and very good. In 
conclusion, the validity and reliability for the 13 item tutor evaluation questionnaire was 
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found to be evident, with reported reliability coefficients and factor analysis testing. This 

was generalizable to groups of 10 students, where group sizes were maintained. 

These findings compared equivocally with those of Wilkerson, Hafler, and Liu (1992) 

and Schmidt & Moust (1995), investigating the tutor behaviors which stimulate learning 

with students. Schmidt & Moust (1995) found that a tutor should use terminology adapted 

from the students’ level of competence; asking questions in a manner the student can 

understand. The tutor’s interest in students’ daily lives and personalities also appeared as 

an important feature of effective tutor behavior. Wilkerson et al. (1992), likewise, 

concluded that two factors describing the skills perceived as most helpful by both tutors 

and students, were maintaining positive interactions within the group and providing 

assistance in getting the work of the group accomplished. 

Des Marchais and Chaput (1997), validated and identified 8 specific tutor tasks as part 

of an undertaking to develop a comprehensive tutor training system at the University of 

Sherbrooke. The 8 tutor task items were as follows: 

1. Manage the PBL method, 2. Facilitate the functioning of a small group, 3. Guide 
the study of specific contents, 4. Favor autonomy, 5. Favor Humanism, 6. Stimulate 
motivation, 7. Evaluate learning, 8. Collaborate in the administration of the study 
program, (p 70) 

The researchers further defined each criteria into competencies for tutors to complete a 
self-assessment of their current and expected level for future tutorial functioning. This 
allowed the tutor to self-regulate, develop and grow as a faculty PBL tutor in a learning 
community. 

Allyson Hadwin (1996), analyzed and interpreted qualitative data from a PBL group in 
a faculty of medicine, with 29 medical students in both focus groups and interviews. In a 
semi-structured interview, the researcher gleaned that there seemed to be a distinct 
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difTerence between, ‘‘going through the motions of PBL and really engaging in PBL” (p 26). 
The researcher surmised embedding the theory and practice of self-regulation in a 
community of learners model. In addition to the “normal” concerns of PBL students, not 
knowing whether or not students have the “right” answer (tolerance for ambiguity) and 
learning how to manage tutorial group processes; the distinction was evident that the role 
of the PBL tutor in making the PBL experience useful or not re-emerged in interviews as 
an important theme. Hadwin (1996) emphasized the following constructs as significant 
implications for the design of PBL learning environments: “1. Content learning is situated 
in relevant activities. 2. Knowledge is socially constructed. 3. Expertise is distributed and 
shared amongst participants. 4. Participants engage in behaviors characteristic of self- 
regulated learning” (pp 12-13). 

Allyson Hadwin (1996) acquiesced with the principles purported by Schmidt et al. 
(1993) viewing the tutor as an “orchestrator” in the learning community. Hadwin 
discerned the tutor as employing the following roles, “(a) The tutor strategically employs 
content and process expertise in the learning practice of the community; and (b) The tutor 
employs content and process expertise through the strategic use of questioning, feedback 
and scaflblding” (p 23) in order to be effective as tutors. The facilitator must guide the 
learning process, not direct the learning. The timing of questioning demands metacognitive 
activity and feedback needs to relate to both content and process expertise for the learner. 
Scaffolding, allows for the novice tutor groups, to be given more direction with a gradual 
development toward total independence in learning (Albanese & Mitchell, 1993). 

Although several resources mentioned items to include, only minimal representative 
samples or examples of PBL faculty and course evaluations were identified in the formal 
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literature. Courts and Mclnemy (1993) best summarized the need and purpose for 
evaluation measures, we fail to learn about our students, then we fail to educate them,” 
furthermore ‘‘assessments inviting interaction, dialogue, reflection and learning offer us a 
way to evaluate, respond and promote learning” (pp 49-50). The researchers strongly 
recommended the portfolio assessment as the clearest, yet most complex, method of 
individual and curricular evaluation. Beecher, Lindemann, Morzinski and Simpson (1997) 
also supported the use of faculty portfolios for reflective practice, promotion and 
development for medical school educators. A customary curriculum vitae format was 
included but expanded upon with evidence presented in the portfolio, such as peer review, 
learner evaluations, performance assessment data and evaluative letters. 

Review of Literature Summary 

At first blush, it appears evaluation of PBL has been explored, but not to an extensive 
depth. More research is essential to authenticate the type, design and content of evaluation 
measures in PBL health care education. Evaluation should be an integral part of the 
learning process and relate to the university mission, philosophy and goals. Evaluation is 
noted to be an essential component in the curriculum improvement process, as well as the 
content and design of evaluation measures. Specific criteria for evaluation measures in 
PBL included, compatibility with the learning objectives and process, assessment of 
resources and shared responsibility for learning. A plethora of content criteria areas in 
PBL evaluation have been considered; these included items for advisement, interpersonal 
relationships skiUs for students and faculty, consideration of instructional methods and 
styles, structure and process, philosophy, knowledge, behavioral and affective 




considerations for students and faculty, tutor roles, and linkage back to the curriculum. 
These shifts in evaluative measures from faculty to student and from content to process. 
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reflect the change in the learning process from passive to active, and from teacher directed 
to learner directed with self evaluation. These educational evaluative measures mirror the 
changes in the practice environment in health care. 

Suggestions from the research for evaluation design criteria, are not nearly as copious 
as those for PBL content criteria. The design choices included attitude scales, portfolios, 
open ended questions, interviews and focus groups or some combination thereof and an 
evaluation that is not too lengthy. Considering some of these aspects for evaluation, while 
in training, is not unlikely and will aid with student preparation for future practice. This 
project addressed the issue of development of appropriate and valid content and design of 
faculty and course evaluation forms, incorporating criteria recommended from the review 
of literature, for NSU PT PBL faculty and students. 
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METHODOLOGY AND PROCEDURES 
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Criteria Establishment 

The objective of this product was to provide evaluation forms for students and faculty 
for Nova Southeastern University Physical Therapy (NSU PT) that were reflective of 
Problem Based Learning (PBL). Standards against which the product was measured for 
the general, content, and design categories, incorporated from the literature review and 
formative committee, were as follows; six items were considered for the general category, 
including: (a) The items are appropriate for student self assessment, (b) the items relate to 
the philosophy and goals of the program, ( c) the items are compatible with the learning 
objectives, (d) the items consider the process of PBL, (e) the items consider resources, and 
(f) the items are reflective of shared responsibility for learning. 

For the content criteria there were ten measurement items: (a) The items are 
appropriate for faculty evaluations, (b) the items are appropriate for course evaluations, 

( c) the items consider advisement , (d) the items consider interpersonal relationships and 
communication skills of both the faculty and students, (e) the items consider instructional 
methods and styles for students and faculty, (f) the items are representative of the process 
and structure of PBL for students and faculty, (g) the items consider philosophy and 
knowledge for students and faculty, (h) the items consider behavioral aspects for students 
and faculty, (i) the items consider specific tutor roles, and (J) the items consider linkage to 
curriculum and life long learning. Finally, in the design category there were four criteria 
items considered: (a) The Likert scale format is suitable for faculty and student evaluations 
in PBL, (b) the length of the evaluation forms is appropriate, ( c) the items provide an 
opportunity for open-ended responses, and (d) the items are clear in wording and concise. 
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Data Gathering and Procedures 

Five procedures were implemented to complete this development practicum. First, a 
review of the literature and review of the NSU and NSU PT Program mission and 
philosophy statements (Appendix A), and review of accreditation documents, was 
conducted. The literature review included theoretical topics of PBL methods, as well as 
applied topics such as development of evaluation forms and sample formats. Further 
considered were content and design criteria specific for the needs of evaluation in PBL in 
health care education. 

Second, an internal formative committee was gathered and criteria were established for 
the faculty and course evaluation forms. The criteria were based on the information from 
the literature review and input from the formative committee. The formative committee 
consisted of the program director, and four faculty members from Nova Southeastern 
University. A complete listing of the five participants on the formative committee and their 
respective affiliation is provided (see Appendix B). The formative committee provided 
initial feedback in response to an open-ended questionnaire for content and format which 
was developed (see Appendix C). 

Third, a draft of the content and design of the evaluation forms and a questionnaire 
was distributed to the summative committee to validate the project. This project draft was 
based on input from the literature review, mission statements of NSU and NSU PT and the 
input of the formative committee gained through the questionnaire. The external 
summative committee consisted of four participants with expertise in PBL in PT. The 
participants and their affiliations are provided (see Appendix B). The questionnaire 
presented to the summative committee was a Likert scale questionnaire which pertained to 

O 
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the sections and topics of the evaluation forms of the project developed. The Likert scale 
questionnaire was provided (see Appendix D) to the committee with a section for open 
ended comments. The summative committee was sent both the draft of the evaluation 
forms project and the questionnaire in order to validate the project. The committee 
members were asked to provide open ended comments on the accuracy and timeliness of 
the forms, for both substance and presentation of the content of the forms, with their 
expert review. The summative committee was provided a two week response time, with 
follow up telephone calls as necessary, to complete the validation process. 

Fourth, data was compiled in tables based on the responses to the questionnaire (see 
Tables 1-3) and the evaluation forms project was revised based upon the summative 
committee input. Fifth, the faculty and course evaluation forms project was presented to 
an authority, the Physical Therapy Program Director and Associate Dean, with 
recommendations for changes and implementation as a result of the questionnaire. 

Assumptions 

An assumption was made that the expert panels evaluation of the content and format of 
the faculty and course evaluation forms was reliable and valid. There was an assumption 
that the experts were able to perform the task accurately. There was an assumption that 
the subject matter experts were able to establish content and face validity through reviews 
by the formative and summative committees. It was also assumed that the results of the 
study were valid for the NSU PT program from an environmental perspective. 

Limitations 

The product was limited in that it was specific to the needs of NSU PT. Another 
potential limitation was whether or not the product forms were generalizable to NSU PT 
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due to unique environments. 
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Chapter 4 
RESULTS 

Evaluation forms were developed and validated for faculty and course evaluations for 
students for Nova Southeastern University (NSU) problem-based learning (PBL) Physical 
Therapy (PT) . The product was developed by conducting a literature review and input 
from a formative committee to determine the appropriate and valid content and design 
criteria for the evaluation product Summative and formative committees were utilized to 
develop and validate the product. The recommendations and changes suggested by the 
formative committee are shown in Table 1. 

Table 1. 

Summary of Formative Committee Questionnaire 



Item 

1. What items do you consider appropriate for 
inclusion on faculty and course evaluation forms 
for utilization at the midterm and Hnal portion of 
the semesters that reflects PBL? 

2. What type of design do you consider most 
appropriate for faculty and course evaluation forms 
which will be reflective of PBL? 

3. How many and what type of items should be included 
in faculty and course evaluations to reflect PBL in a PT 
curriculum with Master’s level students? 

4. Should more than one type of form be utilized to reflect 
PBL for faculty and course evaluations in a PT curriculum? 



Responses 

Team work, self assessment, 

PBL process, learning issues, 
management of PBL, group leadership, 

Likert scale, Thurstone scale, 
paper/pencil form, open ended if not too 
many 

Not more than 10-15, no more than 10 
per form, both process and content 

Eval students (self)^ faculty, team tutor, 
facilitator, labs, confusing - divide up 
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(table continues) 



Item 



Responses 



S. What type of evaluation/assessment scale should he 



Semantic differential, Likert scale. 



utilized in the faculty and course evaluation forms to 



Thurstone, no scale, agree reflect PBL? 



6. Are there any other considerations that need to he 



This is very confusing, an arduous task. 



addressed in the faculty and course evaluation forms for 



good luck, get student input, review other 



PBL in a PT curriculum? 



other PBL forms, let me know if you 



want help 



All formative committee suggestions were taken into consideration as the forms were 
developed. Multiple evaluation forms were necessary to account for all aspects of the 
program. Therefore, as outlined by the formative committee, the following five evaluation 
forms were developed: student self assessment for the course and lab, student performance 
for the clinic, and faculty tutor for the course and lab. Due to the large numbers of 
students (100 per class) the number of items were kept under 10 in most cases and under 20 
for the faculty tutor for the course, as this is the faculty member students spend the most 
time with in the NSU PBL program. Also, this particular form contained more inclusion 
criteria items than the others. This accounted for the formative committee’s 
recommendation to keep the number of evaluation items limited in number. 

The forms each consisted of Likert scale data and open ended comment section as per 
the committee’s suggestions. Thurstone scales were not incorporated, as the forms only call 
for one method and the Likert scale was considered less complex and more definitive. All 
items recommended specifically for appropriate inclusion by the summative committee 
were incorporated in the evaluation forms project. The forms were designed to be 
completed in an anonymous fashion as per committee recommendation, to allow for the 
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student expression without the potential of the evaluation affecting the course grade 
outcome, due to unforeseen bias. 

The changes, recommendations and specific item criteria remarks and editing 
suggestions are displayed in Tables 2 and 3. The summative committee comprised of four 
members validated the product. The validation instrument consisted of 10 criteria related 
to content, 5 criteria for design, and 6 general criteria for evaluation considerations. The 
rating scale used was (5) strongly agree, (4) agree, (3) no opinion, (2) disagree, and (1) 
strongly disagree. If at least two of the four summative committee members had judged 
below a 3 on any item of the content and design validation instrument, the item was 
reviewed and potentially revised. If there was a mean rating of 3.5 on each item of the 
validation instrument, the product was validated. 

The validation revealed a mean rating of 3.5 or greater on all items, except number 9 in 
the content area and criteria number 5 in design, both were less than 3.5. These items were 
addressed with expansion of tutor role criteria and editing of wording, addressing all 
comments in the open ended results section (Table 3). 

Table 2. 

Summary Summative Committee Questionnaire 

rating scale: 5 = Strongly Agree 4= Agree 3 = No opinion 2 = Disagree 1 = Strongly Disagree 

Criteria Statement Response Numbers 

5 4 3 2 1 

Content 

1. The items included are appropriate for 

faculty evaluations 2 2 0 0 0 




(table continues) 
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Criteria Statement Response Numbers 







5 


4 


3 


2 


1 


2. 


The items included are appropriate for 














course evaluations 


0 


3 


0 


1 


0 


3. 


The items consider advisement 


0 


4 


0 


0 


0 


4. 


The items consider interpersonal relationships 














for both faculty and students 


2 


2 


0 


0 


0 


5. 


The items consider instructional methods and 














styles for student and faculty 


1 


2 


0 


1 


0 


6. 


The items are representative of the process and 














structure of PBL for students and faculty 


2 


1 


0 


1 


0 


7. 


The items consider philosophy and knowledge 














for students and faculty 


0 


3 


0 


1 


0 


8 . 


The items consider behavioral aspects for 














students and faculty 


2 


2 


0 


0 


0 


9. 


The items consider specific tutor roles 


0 


2 


0 


2 


0 


10. 


The items consider linkage to curriculum 














and life long learning 


2 


2 


0 


0 


0 


Design 














1. 


The Likert format is suitable for 














student and faculty evaluations 


2 


2 


0 


0 


0 


2. 


Tbe Likert format is suitable for course evals 


1 


3 


0 


0 


0 


3. 


The length of the forms is appropriate 


2 


1 


0 


1 


0 


4. 


The items provide an opportunity for open 














ended responses 


4 


0 


0 


0 


0 
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Criteria Statement 



Response Numbers 
5 4 3 2 1 



5. The items are clear in wording and concisely 
stated 



0 0 0 



General Evaluation 

1. The items are appropriate for student self 
assessment 

2. The items relate to the philosophy and goals of the 
program 

3. The items are compatible with the learning objectives 

4. The items consider the process of PBL 

5. The items consider resources 

6. The items are reflective of shared responsibility for 
learning 



3 

3 
2 

4 



1 

0 

1 

0 

0 



0 

1 

1 

0 

0 



Other General Comments: 

(a) font size too small; (b) headings unclear; (c) confusing and overlapping; (d) nice work; (e) looks good; (f) 
double names; (g) add an N/A category; (h) here you go, good job; (i) scale confusing; (j) student names on 
form or anonymous?; (k) 1 don^t understand the V.A.S. at the bottom, needs instructions on what the 0-5 
stands for; (1) will this page be used or scan forms?; (m) using the words assessment and evaluation is 
redundant 



Table 3 summarizes the open ended responses from the summative committee in 
relation to each four of the five specific evaluation forms, faculty evaluation form for course 
and lab, student evaluation form for course and lab; however no comments were received 
for the student clinical performance evaluation. 







Table 3. 



Specific Open Ended Comments for Evaluation form Item 



Item Number Recommendations and Comments 



Faculty Team Assessment Form 

1 Add sessions to end of the item 

10 Is it only appropriate for mentor? 

11 Add when advisement was provided to the end; 

Not sure 1 like this-what if someone did not need help-why ask it? 

12 Should read when appointments were scheduled; Delete N/A 
Lab Faculty Assessment Form 

3 Don’t necessarily understand; Is this question one they can really answer? Lab 

variety is directly due to their organization and decisions prior to class, they make 
the lab more or less varied, depending on what skills they choose to practice or 
learn about; Change this to lab activities are varied to meet the objectives; 

9 Add lab activities are clinically relevant; Add applicable to clinical 

setting/experience 

Student Self Assessment Course Form 

N/A Add #1 and #7 from lab faculty eval form to this one 

4 Change assess to appraise 

5 Add assess and provide feedback for; Add gives constructive feedback; Change 

assess to appraise 

Student Self Assessment Lab Form 

2 Change this to the 

4 Would be N/A for first year students; Not appropriate for first year, how is this 



accounted for in outcome? 



(table continues) 
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Item Number Recommendations and Comments 



5 Change study to studies; Are all these designed for individual or case study? 

6 Not appropriate for first year, how is this accounted for in outcome? 



The summative committee comments were all considered and all editorial type 
comments were accounted for in the item revisions. Some items were revised, others 
deleted, as suggested, and a few were added per recommendations of the summative 
committee. Other items which were controversial within the summative committee 
comments and item ratings, were left as developed, until the first dissemination of the 
evaluation forms, which served as a pilot for these debatable evaluation items. The 
suggestion to incorporate lab items into the course evaluation for students, was deemed 
inappropriate because these are two distinctly different types of evaluations, one for 
laboratory activities and one for course activities. 

The recommendation to change verbiage from assess to appraise on several items was 
not incorporated because it was believed the item was more meaningful using language 
from the PT clinical decision making model, rather than introduce novel terminology. The 
pilot distribution of the evaluation addressed whether or not this was a viable option. The 
recommendation to add the verbiage to some items was negated, due to merely lengthening 
the item and hence the form, without increasing the ability to interpret the item criteria to 
any greater degree. However, some items were evaluating untapped criteria and therefore 
were included as new items. To account for the fact that some items were not necessarily 
appropriate for first year students, the option of having no feeling either way on the Likert 
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scale was a suitable choice, however some students with advanced experience or other 
appropriate career experiences were able to rate these items, therefore the items were 
maintained. 

To develop the final product, these noted revisions were made and the product was 
formatted for a scannable computer evaluation form with the NSU academic support 
services. During the process of revisions, although font size was reduced significantly for 
the computer forms, it was necessary to split some items into two and sometimes three 
distinct items secondary to physical space limitations on the forms. This accounted for 
some items exceeding the recommended number of items for the evaluation forms. 
Evaluation forms were then proofread and final printing was completed. Forms were then 
presented to the PT Program Director and Associate Dean for final approval for 
implementation. 
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Chapter 5 

DISCUSSION, CONCLUSIONS, IMPLICATIONS AND RECOMMENDATIONS 

Discussion 

The underlying problem addressed by this study was that the Nova Southeastern 
University Physical Therapy (NSU PT) curriculum had been altered from the traditional 
method to Problem Based Learning (PBL) and faculty and course evaluation forms 
required revision to reflect this change in androgogy. The results addressed this problem, 
providing a suitable and valid product of five evaluation forms for the various aspects of 
NSU PT PBL evaluation. 

This project development and validation corroborated the findings of previous studies 
(Dolman et al, 1995, Eley & Stecher, 1997, Hay, 1996 and Schmidt & Moust, 1995), 
regarding content items and design for evaluation forms, and particularly PBL in the 
health care education arena. Similar to the evaluation of health sciences education 
programs by Schmidt et al. (1993), evaluation was validated as a shared responsibility and 
it was appropriate to include students in the evaluative process. Design types for 
evaluation matched those discussed by Eley and Stecher (1997), and although some items in 
this project were not initially well constructed, the importance of this aspect of item 
construction was discovered, as described by these authors and Davis (1993). Although it 
was impossible to incorporate all evaluation content criteria described in the research, the 
specifics provided by Banta et al. (1996), Hay (1996) and Astin (1991) were partially taken 
into consideration and affective, behavioral, psychomotor and a few cognitive items were 
validated in the development of this project. 

Congruent with research by Schmidt & Moust (1995), utilizing similar content items 
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and Likert scale evaluation measures for a PBL program, the project developed in like 
fashion. Other content item considerations, such as those identified by Hay (1996) for 
tutor roles, were correspondingly lengthy in comparison with this project. Hay ascertained 
19 items for this one aspect of PBL evaluation, this project initially was lacking in this area 
per formative committee review. However, the final evaluation product for tutor roles, 
surmised 18 facilitator items for evaluation, addressing the formative committees concerns. 
However, one must consider the serendipity of this heightened number of criteria items, in 
light of the committee members involvement as tutors, compared with the other item 
numbers. In contrast to Hay’s PBL PT evaluations, self-assessment was as predominant as 
tutor/faculty assessment in this final product. These results were influenced by of the 
research findings of May et al. (1995), survey of PT clinical educators to identify generic 
abilities, critically important to PT practice. These outcome abilities, notably, commitment 
to learning, interpersonal skills, communication skills, time management and effective use 
of resources, constructive feedback, problem-solving, professionalism, responsibility and 
critical thinking (May et al., 1995, p 4) were incorporated into the final product. These 
content considerations were also in agreement with the study by Des Marchais and Chaput 
(1997) who validated and identified 8 specific tutor tasks for evaluation. 

However, this product is notably dissimilar from the focus groups and qualitative 
studies performed by Hadwin (1996). Although the depth of information gleaned from this 
type of research is enriching and valuable, this method of evaluation is not practical for a 
large number of students in this PT PBL program. Overall, the summative and formative 
committee feedback, was positive in nature, with constructive criticisms for redesign and 
editorial improvements. 




•f>v 



38 



Conclusions 

Following implementation of the procedures for the project, appropriate and valid 
contents and design of faculty and course evaluation forms, that were representative of 
PBL for NSU PT, were developed. Both the formative and summative committees were 
knowledgeable about the PBL philosophy, however committee members may have been 
disadvantaged as this was a developing and novel curriculum. The committee input and 
feedback were invaluable to the product design and recommendations were made without 
reservation. 



Implications 

Faculty and course evaluation forms for the PBL PT program were developed for NSU 
PT students and faculty. It was expected that the evaluation forms would enable students 
to evaluate faculty and PT courses in a PBL curriculum. This form was expected to 
provide consumer feedback for improvement and development of the NSU PBL PT 
curriculum. Pilot testing and the actual consumer use of the product further addressed 
these potential product outcomes. 

In terms of educational practice, the NSU PT department was able to utilize this 
evaluation form for accreditation requirements and curriculum design and development. 
The forms provided the opportunity for students to offer feedback for both self assessment 
and faculty evaluation in tutorial groups and laboratory learning. Future implications for 
longitudinal studies with the faculty and student evaluation forms and appropriate 
feedback, with regard to program progression, in meeting the overall curriculum objectives 
were indicated. West and Watson (1996) best summarized the process as follows: 

Student-centered learning and assessment is the resultant focus of this product. Such a 

customer driven focus enables faculty, with a well designed curriculum, to foster life 







long learning and self development. PBL necessitates that faculty amend and modify 
the curriculum in a rapid fashion to correlate with a dynamically changing external 
environment. Such activities encourage interdepartmental and interdisciplinary 
cultural milieus, creating positive energies for the faculty, student and business 
community, (p 8) 

Recommendations 

The recommendations, which follow, were made as a result of the evaluation forms 
project. It was recommended that the report be disseminated to all faculty and students, 
and the Commission on Accreditation of Physical Therapy Education (CAPTE) evaluation 
team, following review and approval by both the PT Program Director and Associate Dean. 

It was recommended that the evaluation forms be implemented for the Fall semester, 
1995, for NSU PBL PT. This recommendation was made by the researcher to the director. 
The director approved this recommendation and the forms were implemented by the 
researcher, following provision of an inservice, with faculty and students, for 
implementation of the evaluation forms. The Fall semester evaluation was completed, with 
the assistance of academic support services for NSU PT. This served as a pilot study, and 
revision of the forms from the pilot data, was completed by the researcher and director, 
with full implementation of the evaluation forms for NSU PBL PT in the Spring semester. 
These forms were made available, for CAPTE, during the on-site accreditation evaluation. 

Further recommendations included, reviewing the evaluation forms for faculty and 
student development and longitudinal studies utilizing the product responses to track 
program progress yearly; and review of data on an on-going basis to identify and assess 
curriculum and learning gaps that may occur with students. The researcher made these 
recommendations to both the PT Program Director and Associate Dean for initial 
implementation for Spring 1995, with data collection assistance from academic support 
services. It was also recommended, that the use of this product be reevaluated, by the 
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director and faculty, following an academic year of implementation, as a portion of the 
curriculum review process. In addition, results should be shared with other PT faculty, by 
submission of this research abstract, by the researcher, for the national PT conference. 
Further exploration for the use of portfolios was also indicated as a result of the literature 
review. This researcher has been approved to complete this investigation, following 
discussion with the director and faculty curriculum committee. 

Health care is presently a rapidly changing environment. This must be taken into 
consideration insofar as the needs for both the consumer and the student customer in 
health care education. West and Watson (1996) discussed the need for responsiveness to 
change, stating that educational programs must prepare themselves and their students to 
meet the challenge of change (p 2). If the evaluative process is optimized, evaluation 
instruments can be a springboard to creating strategies for success in new environments. 
Educators cannot predict the way in which health care will be provided in the future, yet 
we can teach how to solve problems within institutional frameworks. Therefore, the final 
recommendation provided, as a result of this study, was an evaluation of the results of this 
product to improve the educational program and these results serving as guide, or 
blueprint for further faculty and curriculum development. This recommendation was 
made by the researcher to the director, faculty and students for implementation. This 
process of review should be completed on an ongoing basis as part of program evaluation 
measures and as a portion of the yearly, curriculum retreat, review process each Summer 
semester, conducted with faculty and the director. It was recommended, to attain the 
faculty review portion of these suggestions, the peer faculty review committee complete this 
task, while assessing faculty annual reports each calendar year. 
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Appendix A 

Nova Southeastern University rNSU) Mission Statement 
(1995-96 Health Professions Division catalog) 

NSU provides educational programs of distinction from prekindergarten through the 
doctoral level at times and in locations convenient to students, prepares students for 
leadership roles in business and the professions, encourages research and community 
service, and fosters an atmosphere of creativity and innovation utilizing technology 
where appropriate. Approved by the Board of Trustees, June 22, 1992. 

NSU Health Professions Division Mission Statement 
The mission of the Health Professions Division is to train printary care health 
practitioners in a multi disciplinary setting, especially for under served areas. 

NSU Physical Therapy Program Mission Statement 
The mission of the NSU Physical Therapy Program is to prepare physical therapists as 
primary health care providers. 

Physical therapist as primary health care provider : a physical therapist who is the 
initial, first or principal care provider for the diagnosis and treatment of physical 
impairments, functional limitations and disabilities that result in movement 
dysfunction. 



NSU Physical Therapy Program Philosophy 
We believe the physical therapist—diagnoses, plans, treat and supervises care for 
physical impairments and disabilities that are the result of dysfunction of any body 
system, must be prepared to consider the influence of mind, body and spirit on health in 
order to prevent dysfunction and promote wellness, must anticipate the need for change 
in the profession and health care delivery, must contribute to the body of knowledge of 
physical therapy and be supportive of the collegial search for truth. 

We believe the physical therapy profession-^-is committed to access to health care for all 
people, is essential to primary health care in order to meet the needs of the undeserved 
across their life spans in all strata of society, is dynamic and responsive to the health 
care demands of current society. 

We believe that physical therapy educationr-demands a strong liberal arts foundation, 
including foreign language, to prepare physical therapists for leadership roles in a 
multi-cultural society, occurs best in a creative atmosphere that fosters exchange among 
faculty, students, and practitioners, who are all self-directed and able to function 
independently and in groups, occurs best when the roles of academic and clinical 
faculty are intertwined because they are equally vital to the learning process. 

We believe that physical therapy faculty —must bring together a variety of academic, 
clinical, and life experiences which complement each other and are necessary to provide 
the depth and breadth of learning opportunities essential to the education of the 
generalist physical therapist or primary care practice. 
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We believe that physical therapy graduates— must be committed to serve the profession 
and society through contributions to the advancement of the profession in their search 
for new knowledge and skills as life-long learners and applied scientists, must be 
committed to serve the profession and society in a variety of roles such as primary care 
providers, consultants, advocates for the disabled and change agents. 

NSU Physical Therapy Program Outcomes 

Graduates will be able to— practice with integrity as generalists in primary health care 
in order to serve diverse populations of any age and cultural, socioeconomic, and 
educational status, participate in activities that insure personal and professional 
growth, communicate appropriately with any audience, practice ethically and legally in 
any health care setting, advocate for the disabled, serve as change agents in 
organizations, including legislative bodies, apply the principals of management and 
consultation in any practice setting to assure efficient and effective health care, 
participate in the development of knowledge that advances the profession, teach in any 
role, choose actions that reffect consideration for the consequences of their decisions. 
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Internal Formative Committee 



Catherine Page, PhD, PT 


NSUPT 


Director & Associate Dean 


Mary Blackinton, MS, PT 


NSUPT 


Assistant Professor 


Leah Nof, PhD, PT 


NSUPT 


Professor 


Ian Ross, MBS, MPT 


NSUPT 


Assistant Professor 


Kathy Perry, PhD, PT 


NSUPT 


Professor 



External Summative Committee 



Mary Jo Wisniewski, EdD, PT 
Linda Woodruff, PhD, PT 
Kelly Gowland, BHSc, PT 
Tom Zeller, MPT 



Wheeling Jesuit College PT* 
North Georgia College PT 
McMaster University PT 
Suncoast Rehabilitation, 
NSU Alumnus 



Director 
Director 
Director 
Staff PT 



^Programs in various stages of development seeking accreditation with either Candidacy 
status or application for Candidacy with Declaration of Intent submitted to American 
Physical Therapy Association (APTA). 

(Programs are placed in this status when the APTA Department of Accreditation receives 
official notiflcation of their intent to seek accreditation from the Commission on 
Accreditation of Physical Therapy Education.) 
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Appendix C 



Formative Committee Questionnaire 
PBL Faculty and Course Evaluation 

1. What items do you consider appropriate for inclusion on faculty and course 
evaluation forms for utilization at the midterm and final portion of the semesters that 
reflects PBL? 



2. What type of design do you consider most appropriate for faculty and course 
evaluation forms which will be reflective of PBL? 



3. How many and what type of items should be included in faculty and course 
evaluations to reflect PBL in a PT curriculum with Master’s level students? 



4. Should more than one type of form be utilized to reflect PBL for faculty and 
course evaluations in a PT curriculum? 



5. What type of evaluation/assessment scale should be utilized in the faculty and 
course evaluation forms to reflect PBL? 



6. Are there any other considerations that need to be addressed in the faculty and 
course evaluation forms for PBL in a PT curriculum? 



Thank you for your assistance! 
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Appendix D 

Summative Co mmittee Questionnaire 

Please rate the items below as 5, 4, 3, 2 or 1. Please use the following rating scale: 

5 = Strongly Agree 4 = Agree 3 = No opinion 2 = Disagree 1 = Strongly Disagree 

Statement Response 

Content 



1. The items included are appropriate for 
faculty evaluations 

2. The items included are appropriate for 
course evaluations 

3. The items consider advisement 

4. The items consider interpersonal relationships 
for both faculty and students 

5. The items consider instructional methods and 
styles for student and faculty 

6. The items are representative of the process and 
structure of PBL for students and faculty 

7. The items consider philosophy and knowledge 
for students and faculty 

8. The items consider behavioral aspects for 
students and faculty 

9. The items consider specific tutor roles 

10. The items consider linkage to curriculum 
and life long learning 

Desig n 

1. The Likert format is suitable for 
student and faculty evaluations 

2. The Likert format is suitable for 
course evaluations 

3. The length of the forms is appropriate 

4. The items provide an opportunity for open 
ended responses 

5. The items are clear in wording and concisely 
stated 

General Evaluation 

1. The items are appropriate for student self 
assessment 

2. The items relate to the philosophy and goals 
of the program 

3. The items are compatible with the learning 
objectives 

4. The items consider the process of PBL 

5. The items consider resources 

6. The items are reflective of shared responsibility 
for learning 

Other Comments: 



5 4 3 2 1 

5 4 3 2 1 

5 4 3 2 1 

5 4 3 2 1 

5 4 3 2 1 

5 4 3 2 1 

5 4 3 2 1 

5 4 3 2 1 

5 4 3 2 1 

5 4 3 2 1 

5 4 3 2 1 

5 4 3 2 1 

5 4 3 2 1 

5 4 3 2 1 

5 4 3 2 1 

5 4 3 2 1 

5 4 3 2 1 

5 4 3 2 1 

5 4 3 2 1 

5 4 3 2 1 

5 4 3 2 1 

Thank you for your assistance! 
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Appendix E 

Evaluation Forms Product 

1. Faculty Team Facilitator Course Evaluation 

2. Laboratory Faculty Evaluation 

3. Course (Student Self-Assessment) Evaluation 

4. Laboratory (Student Self-Assessment) Evaluation 

5. Student Performance - Clinic Evaluation 
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Office of Educational Development 
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Faculty Team Facilitator Course Evaluation 
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Team facilitator: 



Course Name & Number: 
Date: 



I STRONGLY AGREE. 



I AGREE. 



I HAVE NOFEEUNGEnHER WAY. 



GENERAL PURPOSE DATA SHEET II 
form no. 7092 1 



I DISAGREE. 



USE NO 2 Pg.NOl. ONLY_ 



I STRONGLY DISAGREE, a 



Shows respect for students diuing the tutorial. 

Respects alternate opinions that are adequately evidence-based. 

Encourages students to cite and evaluate information sources. 

Consistently provides helpful feedback to group and individuals. 

Provides a reasonable degree of student autonomy in managing group dynamics. 

Provides a reasonable degree of student autonomy in allocating tutorial time and learning tasks. 

Provides clear instructions regarding procedures used during the tutorial session. 

Remains focused on the topic and on group interactions during the tutorial period. 

Expects students to locate and evaluate information. 

10. Relationship of classroom activities to learning objectives is clear. 

1 1 . Relationship of examination items to course objectives is clear. 

(DO NOT BUBBLE THISLmE) 

1 3. Provided helpful feedback on overall curriculum performance progress. 2 

14. Helped me manage my life rather than attempting to manage for me when advisement wasptmdsd T 

15. Is a good professional role model. 3 

16. Available and approachable to students. 

17. Your perception of your benefit &om the interaction with your team leader. 

18. Other faculty in the department is available and approachable. 
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NSU Hesith Professions Division 
Office of Educational Development 

College of Allied Health 
Physical Therapy Department 
Laboratory Faculty Evaluation 

Faculty Team facilitator: 

Course Name & Number: 

* 

Date: 



I STRONGLY AGREE. 



I AGREE. 



GENERAL PURPOSE DATA SHEET II 
form no. 70921 



I HAVE NO FEELING Frmro way 



NO, 2 PENCIL ONLY 



I DISAGREE. 



I STRONGLY DISAGREE/^ ^ 



1 . Relationship of laboratoiy activities to learning objectives is clear. 

2. Laboratoiy activities are appropriately challenging. 

3. Laboratoiy activities are varied to meet objectives. 

4. Faculty Facilitator is available for facilitation in laboratoiy experiences. 

5. Faculty Facilitator is effective in facilitating laboratoiy experiences. 

6 . Provides helpful feedback on laboratoiy ph prVmit«f 

7. Uboratoiy insUuctors and assistants were prepared for the sessions, and worked together well, 
o. KclBtionship of ex am i n ation items to course c^jectives is clear. 

9. Laboratory activities are clinically relevant. 
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NSU Health Professions Division 
Office of Educational Development 

College of A 1 1 i e d Health 
Physical Therapy Department 
Course (Student Self-Assessment) Evaluation 

Faculty Team facilitator: 

Course Name & Number: 

Date: 

I. STRONGLY AGREE, e — 



I AGREE. 
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GENERAL PURPOSE DATA SHEET II 
form no. 70921 
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L I am confident 

2. lam confident 

3. I am confident 

4. lam confident 

5. I am confident 

6. lam confident 

7. lam confident 



8. I am confident 

9. lam confident 

10. I am confident 

11. I am confident 
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that I can identify major topics to determine the additional information needed, 
that I can analyze cases to determine the additional information needed, 
that I can identify appropriate sources for the needed information, 
in my ability to function as a member of a tutorial group. 

in my ability to assess my own strengths and weaknesses and needs for improvement, 
in my ability to assess my peers* strengths and weaknesses, 
in my ability to provide constructive feedback for my peers. 

in my ability to present to my peers* their need for improvement. ^ 

that I can use the information from self and peer evaluations to modify my professional behavior.® 
that I can use the information from self and peer evaluations to modify my skills. ‘ ® 

in my ability to integrate ideas learned in case studies and apply them to other cases. 
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3 . Treatment plan and goals are appropriate. 
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4. Performance of treatment is appropriate. 


© 


© 




• -S' 


• © 




S. Student progresses assessment and treatment in a logical manner. 
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6 Student demonstrates appropriate interaction with staff. 
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7. Student has good rapport with patients. 
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8. Student demonstrates critical thinking during evaluation and treatment. 
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9. Student demonstrates appropriate patient positioning and draping. 
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10. Set-up and clean-up of treatment area are appropriate. 
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